{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Ungraded Lab: Cost Function for Logistic Regression\n",
    "\n",
    "## Goals\n",
    "In this lab you will:\n",
    "- In this lab, you will implement the cost function for logistic regression.\n",
    "\n",
    "## Dataset \n",
    "Let's start with the same dataset as before."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "X = np.array([[0.5, 1.5], [1,1], [1.5, 0.5], [3, 0.5], [2, 2], [1, 2.5]])\n",
    "y = np.array([0, 0, 0, 1, 1, 1]).reshape(-1,1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll use a helper function to plot this data. The data points with label $y=1$ are shown as red crosses, while the data points with label $y=0$ are shown as black circles."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEMCAYAAAAoB2Y1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAOCElEQVR4nO3db6yedX3H8c+HtkZbnDzw6BpLe1iyQIhm4E7YXJ1xJTgcBJfFB7rqA7d5njiD0cThOtN2Wx/siZHsj8lJwbF4lBGQxJANJQFkXQZ6CnX8KS6GUSjB9BBDaO1mB3724L5qy+mfc057X9d1n/v7fiUn3Pc51+H3uyF99zq/c93Xz0kEAKjhgr4nAADoDtEHgEKIPgAUQvQBoBCiDwCFEH0AKKST6Nu+yPadtp+2vd/2e7oYFwDweqs7GudmSfcm+bDtN0ha29G4AICTuO03Z9l+i6R9kn4lvBMMAHrVxfLOJZLmJX3V9mO2d9te18G4AIAFujjTn5L0sKTNSR6xfbOkV5J88aRjpiVNS9K6det+/bLLLmt1TgAwbvbu3ftSkonFjusi+r8s6eEkk83z35Z0U5LrTnf81NRU5ubmWp0TAIwb23uTTC12XOvLO0l+LOl525c2n7pa0lNtjwsAOFVXV+98WtJsc+XOM5I+0dG4AICTdBL9JPskLfpjBwCgXbwjFwAKIfoAUAjRB4BCiD4AFEL0AaAQog8AhRB9ACiE6ANAIUQfAAoh+gBQCNEHgEKIPgAUQvQBoBCiDwCFEH0AKIToA0AhRB8ACiH6AFAI0QeAQog+ABRC9AGgEKIPAIUQfQAohOgDQCFEHwAKIfoAUAjRB4BCVncxiO1nJR2W9JqkV5NMdTEuAOD1Ool+43eSvNTheACABVjeAYBCuop+JH3H9l7b0x2NCQBYoKvlnfcmecH22yTdZ/vpJA8d/2LzF8G0JG3cuLGjKQFAPZ2c6Sd5ofnnIUl3S7pqwddnkkwlmZqYmOhiSgBQUuvRt73O9puPP5b0AUlPtD0uAOBUXSzvvF3S3baPj/f1JPd2MC4AYIHWo5/kGUm/1vY4AIDFcckmABRC9AGgEKIPAIUQfQAohOgDQCFEHwAKIfoAUAjRB4BCiD4AFEL0AaAQog8AhRB9ACiE6ANAIUQfAAoh+gBQCNEHgEKIPgAUQvQBoBCiDwCFEH0AKIToA0AhRB8ACiH6AFAI0QeAQog+ABRC9AGgEKIPAIUQfQAopLPo215l+zHb93Q1JgDg9bo8079R0v4OxwMALNBJ9G1vkHSdpN1djAcAOL2uzvS/LOnzkn5+ui/anrY9Z3tufn6+oykBQD2tR9/29ZIOJdl7pmOSzCSZSjI1MTHR9pQAoKwuzvQ3S7rB9rOSbpe0xfbXOhgXALBA69FP8oUkG5JMSvqIpPuTfKztcQEAp+I6fQAoZHWXgyV5UNKDXY4JADiBM30AKIToA0AhRB8ACiH6AFAI0QeAQog+ABRC9AGgEKIPAIUQfQAohOgDQCFEHwAKIfoAUAjRB4BCiD4AFEL0AaAQog8AhRB9ACiE6ANAIUQfAAoh+gBQCNEHgEKIPgAUQvQBoBCiDwCFEH0AKIToA0AhRB8ACjnn6Nv+syUe90bb37P9A9tP2t55rmMCAM7P6qUeaPuOk59KukLS3yzhW38maUuSI7bXSNpj+1+TPLysmQIAztuSoy/plSR/cvyJ7a8s5ZuSRNKR5uma5iPLGBcAMCTLWd7ZteD5tqV+o+1VtvdJOiTpviSPLPj6tO0523Pz8/PLmBIAYDkWjb7tm207yX+f/PkkP1nqIEleS3KFpA2SrrL9zgVfn0kylWRqYmJiqf9aAMAyLeVM/7Ckb9leJ0m2f9f2v5/LYElelvSApGvP5fsBAOdn0TX9JH9h+w8lPWj7mAbr8zctdQDbE5L+L8nLtt8k6Rot7RfAAIAhWzT6tq+W9ElJP5W0XtIfJfnhMsZYL+k226s0+MnijiT3nMtkAQDnZylX72yT9MUke2y/S9I/2/5skvuXMkCS/5R05flMEgAwHEtZ3tly0uPHbX9Q0l2SfqvNiQEAhm/Z78hN8qKkq1uYCwCgZed0G4Yk/zPsiQAA2scN1wCgEKIPAIUQfQAohOgDQCFEHwAKIfoAUAjR79qOHX3PAEBhRL9rO9ktEkB/iD4AFEL0u7Bjh2QPPqQTj1nqAdAxD7awHR1TU1OZm5vrexrtsaUR+28OYOWzvTfJ1GLHcaYPAIUQ/a5t3973DAAURvS7xjo+gB4RfQAohOgDQCFEHwAKIfoAUAjRB4BCiD4AFEL0AaAQog8AhRB9ACiE6ANAIa1H3/bFth+w/ZTtJ23f2PaYAIDT6+JM/1VJn0tyuaTflPQp25d3MC76wL2FgJHWevSTvJjk0ebxYUn7Jb2j7XHRE7aDBEZap2v6ticlXSnpkS7HBQAMdBZ92xdKukvSZ5K8suBr07bnbM/Nz893NSUMC9tBAitGJ9sl2l4j6R5J307ypbMdO/bbJY47toMEejEy2yXatqRbJO1fLPgAgHZ1sbyzWdLHJW2xva/5+L0OxkUf2A4SGGmr2x4gyR5JbnscjAjW8YGRxjtyAaAQog8AhRB9ACiE6ANAIUQfAAoh+gBQCNEHgEKIPgAUQvQBoBCiDwCFlI3+7OysJicndcEFF2hyclKzs7N9TwkAWtf6vXdG0ezsrKanp3X06FFJ0oEDBzQ9PS1J2rp1a59TA4BWlTzT37Zt2y+Cf9zRo0e1bdu2nmYEAN0oGf3nnntuWZ8HgHFRMvobN25c1ucBYFyUjP6uXbu0du3a131u7dq12rVrV08zAoBulIz+1q1bNTMzo02bNsm2Nm3apJmZGX6JC2DsdbIx+nKwMToALN/IbIwOABgdRB8ACiH6AFAI0QeAQog+ABRC9AGgEKIPAIUQfQAohOgDQCGtR9/2rbYP2X6i7bEAAGfXxZn+P0q6toNxRho7dQEYBa3vnJXkIduTbY8zytipC8CoYE2/A+zUBWBUjET0bU/bnrM9Nz8/3/d0ho6dugCMipGIfpKZJFNJpiYmJvqeztCxUxeAUTES0R937NQFYFR0ccnmNyT9h6RLbR+0/cdtjzlq2KkLwKhg5ywAGAPsnAUAOAXRB4BCiD4AFEL0AaAQog8AhRB9ACiE6ANAIUQfAAoh+gBQCNEHgEKI/phip66W7NjR9wyA80L0x9DxnboOHDigJL/YqYvwD8HOnX3PADgvRH8MsVMXgDMh+mOInbqGbMcOyR58SCces9SDFYjojyF26hqyHTukZPAhnXhM9LECEf0xxE5dAM6E6I8hdupq0fbtfc8AOC/snAUAY4CdswAApyD6AFAI0QeAQog+ABRC9AGgEKIPAIUQfQAohOgDQCFEHwAKIfoAUEgn0bd9re0f2v6R7Zu6GBMAcKrWo297laS/l/RBSZdL+qjty9seFwBwqi7O9K+S9KMkzyQ5Jul2SR/qYFwAwAKrOxjjHZKeP+n5QUm/cfIBtqclTTdPf2b7iQ7m1Ze3Snqp70m0iNe3so3z6xvn1yZJly7loC6iv6gkM5JmJMn23FJuD7pS8fpWNl7fyjXOr00avL6lHNfF8s4Lki4+6fmG5nMAgI51Ef3vS/pV25fYfoOkj0j6VgfjAgAWaH15J8mrtv9U0rclrZJ0a5Inz/ItM23PqWe8vpWN17dyjfNrk5b4+kZuu0QAQHt4Ry4AFEL0AaCQkYr+ON+uwfattg+N63sQbF9s+wHbT9l+0vaNfc9pWGy/0fb3bP+geW07+55TG2yvsv2Y7Xv6nsuw2X7W9uO29y310saVxPZFtu+0/bTt/bbfc8ZjR2VNv7ldw39JukaDN3B9X9JHkzzV68SGxPb7JB2R9E9J3tn3fIbN9npJ65M8avvNkvZK+v1x+P9n25LWJTlie42kPZJuTPJwz1MbKtuflTQl6ZeSXN/3fIbJ9rOSppKM5ZuzbN8m6d+S7G6uklyb5OXTHTtKZ/pjfbuGJA9J+knf82hLkheTPNo8Pixpvwbvxl7xMnCkebqm+RiNs6Uhsb1B0nWSdvc9FyyP7bdIep+kWyQpybEzBV8areif7nYNYxGNamxPSrpS0iM9T2VomqWPfZIOSbovydi8tsaXJX1e0s97nkdbIuk7tvc2t30ZJ5dImpf01WZ5brftdWc6eJSijzFg+0JJd0n6TJJX+p7PsCR5LckVGryj/CrbY7NEZ/t6SYeS7O17Li16b5J3a3C33081y63jYrWkd0v6SpIrJf1U0hl/JzpK0ed2DStcs959l6TZJN/sez5taH5sfkDStT1PZZg2S7qhWfe+XdIW21/rd0rDleSF5p+HJN2twXLyuDgo6eBJP33eqcFfAqc1StHndg0rWPPLzlsk7U/ypb7nM0y2J2xf1Dx+kwYXGzzd66SGKMkXkmxIMqnBn7v7k3ys52kNje11zcUFapY9PiBpbK6iS/JjSc/bPn6XzaslnfECipG4y6Z0TrdrWFFsf0PS+yW91fZBSduT3NLvrIZqs6SPS3q8WfuWpD9P8i/9TWlo1ku6rbnC7AJJdyQZu8sax9jbJd09OC/RaklfT3Jvv1Mauk9Lmm1OmJ+R9IkzHTgyl2wCANo3Sss7AICWEX0AKIToA0AhRB8ACiH6AFAI0QeAQog+cBbN7aKvaR7/te2/7XtOwPkYmTdnASNqu6S/tP02DW4id0PP8wHOC2/OAhZh+7uSLpT0/iSHm7fy/4OkY5IeTDLb6wSBZWB5BzgL2+/S4DYMx5p9AiTpDyTdmeST4swfKwzRB86g2Q1sVoPNfI7YPn5nzQ06sffDa33MDThXRB84DdtrJX1T0ueS7Jf0Vxqs70uDW9luaB7zZwgrCmv6wDI1a/p/J+l/Je1hTR8rCdEHgEL40RQACiH6AFAI0QeAQog+ABRC9AGgEKIPAIUQfQAohOgDQCFEHwAK+X8cbcpUdbPOjQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from lab_utils import plot_data\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plot_data(X,y)\n",
    "\n",
    "# Set both axes to be from 0-6\n",
    "plt.axis([0, 6, 0, 6])\n",
    "# Set the y-axis label\n",
    "plt.ylabel('$x_1$')\n",
    "# Set the x-axis label\n",
    "plt.xlabel('$x_0$')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cost function\n",
    "\n",
    "First, you will implement a non-vectorized version of the cost function. Then, you will implement a vectorized version.\n",
    "\n",
    "\n",
    "### Non- vectorized version\n",
    "\n",
    "Recall that for logistic regression, the cost function is of the form \n",
    "\n",
    "$$ J(\\mathbf{w},b) = \\frac{1}{m} \\sum_{i=0}^{m-1} \\left[ cost(f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}), y^{(i)}) \\right] \\tag{1}$$\n",
    "\n",
    "where\n",
    "* m is the number of training examples in the dataset \n",
    "\n",
    "* $cost(f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}), y^{(i)})$ is the cost for a single data point, which is - \n",
    "\n",
    "    $$cost(f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}), y^{(i)}) = -y^{(i)} \\log\\left(f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right) - \\left( 1 - y^{(i)}\\right) \\log \\left( 1 - f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right) \\tag{2}$$\n",
    "    \n",
    "*  $f_{\\mathbf{w},b}(\\mathbf{x}^{(i)})$ is the model's prediction, while $y^{(i)}$, which is the actual label\n",
    "\n",
    "\n",
    "* For a logistic regression model for the dataset given above, the model can be representented as:\n",
    "\n",
    "    $f_{\\mathbf{w},b}(x) = g(w_0x_0 + w_1x_1 + b) = g(\\mathbf{x^Tw} + b)$\n",
    "\n",
    "    where $g(z)$ is the sigmoid function:\n",
    "\n",
    "    $g(z) = \\frac{1}{1+e^{-z}}$ \n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Side Note: sigmoid function implementation\n",
    "We've implemented the `sigmoid` function for you already and you can simply import and use it, as shown in the code block below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.5\n"
     ]
    }
   ],
   "source": [
    "from lab_utils import sigmoid \n",
    "\n",
    "print(sigmoid(0))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='ex-02'></a>\n",
    "#### Exercise \n",
    "\n",
    "Please complete  the `compute_cost_logistic_loop`:\n",
    "- Create a variable outside the loop to store the costs\n",
    "- Loop over all examples in the training set.\n",
    "    - calculate the cost for each training example \n",
    "        - Calculate `z`\n",
    "        - $ z =  w_0x_0^{(i)} + w_1x_1^{(i)} + b = \\mathbf{x}^{(i)T}\\mathbf{w} +b $\n",
    "        - Predict `f` where `g` is the sigmoid function\n",
    "            - $ f =  g(z) $\n",
    "        - Calculate the cost for each example  \n",
    "            $cost =  (-y^{(i)} * \\log f) - ((1 - y^{(i)})\\log( 1 - f))$\n",
    "     - Add this cost to the total cost variable created outside the loop   \n",
    "- Get the sum of cost from all iterations and return the total divided by the number of examples.\n",
    "\n",
    "\n",
    "As you are doing this, remember that the variables X_train and y_train are not scalar values but matrices of shape ($m, n$) and ($𝑚$,1) respectively, where  $𝑛$ is the number of features and $𝑚$ is the number of training examples.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "  <summary><font size=\"2\" color=\"darkgreen\"><b>Hints</b></font></summary>\n",
    "    \n",
    "```python     \n",
    "def compute_cost_logistic_loop(X, y, w, b):\n",
    "    \"\"\"\n",
    "    Computes the cost over all examples\n",
    "    Args:\n",
    "      X : (array_like Shape (m,n)) data, m examples by n features\n",
    "      y : (array_like Shape (m,1)) target value \n",
    "      w : (array_like Shape (n,1)) Values of parameters of the model      \n",
    "      b : (array_like Shape (n,1)) Values of bias parameter of the model\n",
    "    Returns:\n",
    "      total_cost: (scalar)         cost \n",
    "    \"\"\"\n",
    "\n",
    "    m, n = X.shape\n",
    "    cost = 0.\n",
    "    for i in range(m):\n",
    "        ### BEGIN SOLUTION ###  \n",
    "        z = 0\n",
    "        for j in range(n):\n",
    "            z = z + w[j] * X[i][j]\n",
    "        z = z + b\n",
    "        f = sigmoid(z)\n",
    "        \n",
    "        cost +=  -y[i]*np.log(f) - (1-y[i])*np.log(1-f)\n",
    "        ### END CODE HERE ### \n",
    "             \n",
    "    total_cost = (1/m) * cost\n",
    "    return total_cost[0] \n",
    "```\n",
    "</details>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def compute_cost_logistic_loop(X, y, w, b):\n",
    "    \"\"\"\n",
    "    Computes the cost over all examples\n",
    "    Args:\n",
    "      X : (array_like Shape (m,n)) data, m examples by n features\n",
    "      y : (array_like Shape (m,1)) target value \n",
    "      w : (array_like Shape (n,1)) Values of parameters of the model      \n",
    "      b : (array_like Shape (n,1)) Values of bias parameter of the model\n",
    "    Returns:\n",
    "      total_cost: (scalar)         cost \n",
    "    \"\"\"\n",
    "\n",
    "    m, n = X.shape\n",
    "    cost = 0.\n",
    "    for i in range(m):\n",
    "        ### START CODE HERE ### \n",
    "        ### BEGIN SOLUTION ###  \n",
    "        z = 0\n",
    "        for j in range(n):\n",
    "            z = z + w[j] * X[i][j]\n",
    "        z = z + b\n",
    "        f = sigmoid(z)\n",
    "        \n",
    "        cost +=  -y[i]*np.log(f) - (1-y[i])*np.log(1-f)\n",
    "        ### END SOLUTION ###  \n",
    "        ### END CODE HERE ### \n",
    "             \n",
    "    total_cost = (1/m) * cost\n",
    "    return total_cost[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check the implementation of your cost function using the cell below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.3668667864055174\n"
     ]
    }
   ],
   "source": [
    "# note, w is defined as a column vector\n",
    "w = np.array([1,1]).reshape(-1,1)\n",
    "b = -3\n",
    "print(compute_cost_logistic_loop(X, y, w, b))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Expected output**: 0.3668667864055174"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's see what the cost function output is for a different value of $w$. \n",
    "\n",
    "* In the last ungraded lab, you plotted the decision boundary for  $b = -3, w_0 = 1, w_1 = 1$. That is, you had `w = np.array([-3,1,1])`.\n",
    "\n",
    "* Let's say you want to see if $b = -4, w_0 = 1, w_1 = 1$, or `w = np.array([-4,1,1])` provides a better model.\n",
    "\n",
    "Let's first plot the decision boundary for these two different $b$ values to see which one fits the data better.\n",
    "\n",
    "* For $b = -3, w_0 = 1, w_1 = 1$, we'll plot $-3 + x_0+x_1 = 0$ (shown in blue)\n",
    "* For $b = -4, w_0 = 1, w_1 = 1$, we'll plot $-4 + x_0+x_1 = 0$ (shown in magenta)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAEWCAYAAACHVDePAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAwt0lEQVR4nO3dd3xV9f3H8dcng4SQQDBsAgkQRgYhGweouFfFUusoKg5ELZZirbWWKjjwp9YqVFsriooKLhx114EoDshNSIAMNgQShmEEAiEQku/vj3NjozIy7s3JPffzfDzyMLk595zPsfV9v/mecz5fMcaglFLK2QLsLkAppZT3adgrpZQf0LBXSik/oGGvlFJ+QMNeKaX8gIa9Ukr5AQ171eaJyEciMq4R2+0Tkf6tUZNdRGSjiJxldx3K9wTZXYByBhHZCHQHDgO1QBHwIjDLGFPXkn0bY85v5HbhLTnO0TQ4t1qgBvgWuNkYs9kbx1PKG3RkrzzpF8aYCCAGeAi4E5htb0ke8wv3h0lPYDvwhM31NImI6MDOz2nYK48zxuwxxrwLXA6ME5EkABEJEZFHRWSTiGwXkX+LSPv694nIaBHJF5G9IrJORM5zv75QRMa7v48TkS9FZI+I7BCR1xq834hInPv7TiLyooiUi0iJiPxVRALcv7tWRL5217JbRDaISGP/eqgG5gMJDY57rGNNE5GXG2wb664zqMG53S8i34hIpYh8IiJdGmx/tXufO0VkSsNaRCRLRL4TkQoR2SoiT4pIu5/8+5goImuANSLyTxH5+0/28a6I3NaYc1e+TcNeeY0xJhsoBUa6X3oIGASkAHFAb+AesIILa9rnDiASOBXYeITd3g98AnQGojn6CPsJoBPQHzgNuAa4rsHvhwOrgC7AI8BsEZHjnZOIhGF9iC1uwrGO5zfu7bsB7YA/uo+VADwFXA30AqKwzrleLXCb+xxOAs4EfvuTfV+Cda4JwBzgygYfRF2As4B5TahV+SgNe+VtW4AT3EE6AbjNGLPLGFMJPAhc4d7uBuA5Y8ynxpg6Y0yZMWblEfZXgzVN1MsYU22M+fqnG4hIoHu/dxljKo0xG4G/Y4VmvRJjzDPGmFqsEOyJNS9/NO+ISAWwBzgb+FsTjnU8zxtjVhtjDgCvY30YAlwKvG+M+coYcxC4G/jh+ocxJtcYs9gYc9h93KexPmwa+j/3v+8D7g/fPVgfCrjrXmiM2d6EWpWP0rBX3tYb2AV0BcKAXPe0QwXwsft1gD7Aukbs70+AANkiUigi1x9hmy5AMFDS4LUSdy31ttV/Y4ypcn97rAu8lxhjIoFQ4FbgSxHp0chjHc+2Bt9XNaijF/DDRWBjzH5gZ/3PIjJIRN4XkW0ishfrw7MLP/bTi8hzgKvc318FvNSEOpUP07BXXiMimVih9zWwAzgAJBpjIt1fnRrcQbMZGHC8fRpjthljbjTG9AJuAv5VP0/fwA7+9xdAvb5AWcvOCIwxtcaYt7CmUEY04lj7sT7k6vVowuG2Yn0IAj9MIUU1+P1TwEpgoDGmI/AXrA/CH5X8k59fBkaLyDAgHninCfUoH6ZhrzxORDqKyEXAq8DLxpgV7tsvnwEeF5Fu7u16i8i57rfNBq4TkTNFJMD9uyFH2PevRaR+3no3Vpj96NZO99TM68B0EYkQkRjgD1hB19JzExEZjXXNoLgRx8oHThWRviLSCbirCYebD1wkIiPcF17v48f/zUYAe4F97n9Xtxxvh8aYUsCFNaJ/0z11pPyAhr3ypPdEpBJrlD4FeIwfX6i8E1gLLHZPO3wGDIYfLuZeBzyONa/8JT8eLdfLBJaIyD7gXeD3xpj1R9jud1ij6vVYf1nMA55r4bntwwrX6cA4Y0zh8Y5ljPkUeA1YDuQC7zf2gO79T3TvbyvWh1tpg03+iHVxtxLrg/S1n+7jKOYAQ9EpHL8iuniJUv5FRE7F+ssjxmgA+A0d2SvlR0QkGPg98KwGvX/xatiLSKSIzBeRlSJSLCInefN4SqmjE5F4oALrNtMZthajWp1Xp3FEZA6wyBjzrPsCU5gxpsJrB1RKKXVEXgt7950H+UB//XNRKaXs5c3mSP2AcuB59z29uVh3TuxvuJGITMB6spIOHTqkDxnys7vtlFJKHUVubu4OY0zX423nzZF9Blb/kFOMMUtEZCaw1xhz99Hek5GRYXJycrxSj1JKOZGI5BpjMo63nTcv0JYCpcaYJe6f5wNpXjyeUkqpo/Ba2BtjtgGbRWSw+6UzsRa0UEop1cq8vaDB74C57jtx1tO0tq9KKaU8xKthb4zJB447l6SUUk1RU1NDaWkp1dXVdpfSakJDQ4mOjiY4OLhZ79elypRSPqe0tJSIiAhiY2NpxJozPs8Yw86dOyktLaVfv37N2oe2S1BK+Zzq6mqioqL8IugBRISoqKgW/SWjYa+U8kn+EvT1Wnq+GvZKKeUHNOyVUsoPaNgrpZQf0LBXSqlmWrhwIVdffXWL9lFdXU1WVhbDhg0jMTGRqVOneqi6H9NbL5VSqpmWLVtGampqi/YREhLCggULCA8Pp6amhhEjRnD++edz4okneqhKi47slVKqmfLz8ykrK2P48OH079+fhQsXNnkfIkJ4eDhgPSxWU1PjlTuNdGSvlPJpkydDfr5n95mSAjNmHH+7ZcuWcfHFF7NkyRI++eQT7r77bhYtWsTIkSOprKz82faPPvooZ5111s9er62tJT09nbVr1zJx4kSGDx/e8pP4CQ17pZRqhpqaGnbs2MFf/vIXAFJSUtixYwcAixYtatK+AgMDyc/Pp6Kigl/+8pcUFBSQlJTk0Xo17JVSPq0xI3BvWLlyJXFxcbRr1w6ApUuXMmzYMIBjjuxXrVrFM888A8CHH35Ir169fvh9ZGQko0aN4uOPP9awV0qptiA/P58NGzZw8OBBampquPfee3n88ceBY4/szzrrLCZOnPjDz+Xl5QQHBxMZGcmBAwf49NNPufPOOz1er4a9Uko1w7JlyxgzZgwnn3wyBw4c4O67727WHTRbt25l3Lhx1NbWUldXx2WXXcZFF13k8Xo17JVSqhkeffRRj+wnOTmZvLw8j+zrWPTWS6WU8gMa9kop5Qc07JVSyg9o2CullB/QsFdKKT+gYa+UUn5Aw14ppfyAhr1SSvkBDXullGomTyxeUq+2tpbU1FSvPD0LGvZKKdVsnli8pN7MmTOJj4/3yL6ORMNeKaWayROLlwCUlpbywQcfMH78eM8W2ID2xlFK+bQ1k9ewL3+fR/cZnhLOwBkDj7udpxYvmTx5Mo888sgR3+MpXg17EdkIVAK1wGFjTIY3j6eUUq3FU4uXvP/++3Tr1o309PRm/2XQGK0xsh9ljNnRCsdRSvmhxozAvcFTi5d88803vPvuu3z44YdUV1ezd+9errrqKl5++WXPFmyM8doXsBHo0tjt09PTjVJKHU9RUZHdJZgXX3zRxMbGmurqalNZWWlOPPFE891337Von1988YW58MILj/r7I503kGMaka/eHtkb4BMRMcDTxphZXj6eUkq1Ck8tXtJavB32I4wxZSLSDfhURFYaY75quIGITAAmAPTt29fL5SillGd4avGShk4//XROP/10j+8XvHzrpTGmzP3P74G3gawjbDPLGJNhjMno2rWrN8tRSim/5bWwF5EOIhJR/z1wDlDgreMppZQ6Om9O43QH3haR+uPMM8Z87MXjKaWUOgqvhb0xZj0wzFv7V0op1XjaLkEppfyAhr1SSvkBDXullGpFTz75JHFxcYjID+0VWoOGvVLKf0ybZncFnHLKKXz22WfExMS06nE17JVS/uPeez22q3vuuYcZM2b88POUKVOYOXPmcd+XmppKbGysx+poLG1xrJRSzXD99dczZswYJk+eTF1dHa+++ioLFiwgJSXliNvPmzePhISE1i2yAQ17pZSzTZv24xG99ewPTJ3aommd2NhYoqKiyMvLY/v27aSmphITE0N+fn5LqvUaDXullLNNm/a/UBcBqyOvR4wfP54XXniBbdu2cf3111NZWcnIkSOPuK2O7JVSykf98pe/5J577qGmpoZ58+YRGBjYZkf2eoFWKeU/pk716O7atWvHqFGjuOyyywgMDGzUe/7xj38QHR1NaWkpycnJXl13tiEd2Sul/IeHb72sq6tj8eLFvPHGG41+z6RJk5g0aZJH62gMHdkrpVQzFBUVERcXx5lnnsnAgfYsjdgUOrJXSqlmSEhIYP369XaX0Wg6sldKKT+gYa+UUn5Aw14ppfyAhr1SSvkBDXullGpFGzZsYPjw4cTFxXH55Zdz6NChVjmuhr1SyvHmzp1LbGwsAQEBxMbGMnfuXNtqufPOO7nttttYu3YtnTt3Zvbs2a1yXA17pZSjzZ07lwkTJlBSUoIxhpKSEiZMmNDiwG9Oi2NjDAsWLODSSy8FYNy4cbzzzjstqqOx9D57pZSjTZkyhaqqqh+9VlVVxZQpUxg7dmyz99ucFsfdunUjMjKSoCAreqOjoykrK2t2DU2hYa+UcrRNmzY16fXGak6L49ZchvCnNOyVUo7Wt29fSkpKjvh6SzW1xXF8fDwVFRUcPnyYoKAgSktL6d27d4vraAwNe6WUo02fPp0JEyb8aConLCyM6dOnt3jfzWlxPGrUKObPn88VV1zBnDlzGD16dIvraAy9QKuUcrSxY8cya9YsYmJiEBFiYmKYNWtWi+br6zWnxfHDDz/MY489RlxcHDt37uSGG25ocR2NoSN7pZTjjR071iPh/lPNaXHcv39/srOzPV7L8ejIXimlmkFbHCullB/QFsc/ISKBIpInIu97+1hKKf9hPLhwuC9o6fm2xjTO74HiVjiOUspPhIaGsnPnTr8JfGMMO3fuJDQ0tNn78Oo0johEAxcC04E/HG/7Q9sPUXe4joAgvZSglDq6+gW7y8vL7S6l1YSGhhIdHd3s93t7zn4G8Ccg4mgbiMgEYALAIAaRd1Ieg2cPJjw53MulKaV8VXBwMP369bO7DJ/itSG0iFwEfG+MyT3WdsaYWcaYDGNMRvv+7akuqSY3PZcNUzdQd7DOW+UppZRf8eZ8ySnAxSKyEXgVOENEXj7WG4I6B5FVnEW3K7pRcl8JOWk57Fm8x4slKqWUf/Ba2Btj7jLGRBtjYoErgAXGmKuO977gqGDiX4pn6AdDqd1bS97Jeay9bS21+2u9VapSSjlem70SGnVBFJmFmfS6pRelM0pxDXWx+/PddpellFI+qVXC3hiz0BhzUVPfF9QxiEH/HETKlylIkLDsrGWsHL+Smooab5SplFKO1WZH9g1FnhpJxrIM+tzZh20vbMOV4GLHf+zrC62UUr7GJ8IeILB9IAMeGkD6knSCuwVTcEkBhZcXcmh76yzWq5RSvsxnwr5eRHoE6a50+j3Qjx3v7CA7IZttL23zmyfplFKqOXwu7AECggOImRJDRn4GYYPDWHnNSlZcuILqTdV2l6aUUm2ST4Z9vQ7xHUhdlErczDgqvqzAleii7F9lmDod5SulVEM+HfYAEihET4omszCTjid1ZM3ENeSfnk/Vqqrjv1kppfyEz4d9vfax7Un+bzKDnx/M/hX7cQ1zUfJQCXWHteWCUko5JuwBRISe1/YksyiTqAui2HDXBpYOX0plfqXdpSmllK0cFfb1QnqGkPRWEonzEzlYdpDcjFzWT1lPbbW2XFBK+SdHhn29rr/qSlZRFt2v6s6mBzeRm5rLnm+1sZpSyv84OuwBgk8IJv6FeJI/Tqa2qpa8EXmsmbSGw/sO212aUkq1GseHfb0Tzj2BzIJMek/sTdmTZbiSXOz6ZJfdZSmlVKvwm7AHCIoIYuATA0n5KoWA0ACWn7ucldetpGaXNlZTSjmbX4V9vcgRkWTkZ9D3rr5se2kb2QnZlL/pP2tZKqX8j1+GPUBgaCD9H+xPuiudkJ4hFF5aSMGlBRzcdtDu0pRSyuP8NuzrRaRGkJadRr8H+7Hz/Z24ElxsfWGrNlZTSjmK34c9uBur3eVurJYQxqrrVrH83OUc2HjA7tKUUsojNOwb6DCkA6lfpTLwyYHs/W4vriQXpU+UamM1pZTP07D/CQkQek/sTWZBJp1GdGLtpLXkjcxjf/F+u0tTSqlm07A/itCYUJI/SmbInCFUFVeRk5JDyYMl1NVoYzWllO/RsD8GEaHHNT3IKs6iy8Vd2DBlA7mZuVQu1cZqSinfomHfCO26tyPxjUQS30ykZnsNuVm5rPvzOmoPaGM1pZRv0LBvgq5jupJZlEmPcT3Y/PBmclJyqFhUYXdZSil1XBr2TRTcOZghs4eQ/Gky5pAh/9R8Vk9czeFKbaymlGq7NOyb6YSzTiBjRQa9f9+bLU9twZXoYudHO+0uSymljkjDvgWCwoMYOGMgqd+kEhgeyIoLVlB8TTE1O7WxmlKqbdGw94BOJ3UiIy+DmL/G8P0r35Mdn833r3+vLReUUm1Gs8JeRM5uxDahIpItIstEpFBE7m3OsXxFQEgA/e7vR3pOOiF9Qyi6vIjCMYUc3KKN1ZRS9mvuyH52I7Y5CJxhjBkGpADniciJzTyezwgfFk7a4jT6P9KfXR/vIjshm62ztbGaUspeQUf7hYi8e7RfAVHH27Gx0m2f+8dg99cxE2/fvmP91ncEBAXQ946+dBndhVU3rmLV+FVsf2U7g2cNpn3/9naXp5TyQ3K0EaeI7Aau4n+B/cOvgNeMMd2Pu3ORQCAXiAP+aYy58wjbTAAmWD+lp0+alMP06RAe3oSzaMNMnWHLrC2s/9N6TK2h3/R+RP8uGgkUu0tTSjmAiOQaYzKOu90xwv4j4BFjzBdH+N1XxphTm1BMJPA28DtjTMHRtuvWLcOUl+cQGwuzZsHZx70y4DuqN1ez+qbV7PpoFxHDIxgyewgdEjvYXZZSysc1NuyPOmdvjDnfGPOFiCQc4df3NKUYY0wF8AVw3rG269sXFi2CkBA45xy4/nrYvbspR2q7QvuEMvSDocS/HM+BtQfISc1h4/0bqTukjdWUUt7XmAu0r4vInWJpLyJPAP93vDeJSFf3iB4RaQ+cDaw83vtGjID8fLjrLnjxRUhIgLfeakSVPkBE6D62O1lFWXQZ04WN92wkNyOXva69dpemlHK4xoT9cKAP8C3gArYApzTifT2BL0Rkuft9nxpj3m9MUaGh8OCD4HJBjx7wq1/BpZfCtm2NeXfb165bOxJfTSTpnSRqdtaw9MSlrPvTOmqrtLGaUso7GhP2NcABoD0QCmwwxhx37sEYs9wYk2qMSTbGJBlj7mtqcampkJ1tBf/771uj/DlzwCl3MXYZ3YXMwkx63tCTzX/bTM6wHCq+rLC7LKWUAzUm7F1YYZ8JjASuFJE3vFpVA8HB1pROfr4V9tdeC+efDyUlrVWBdwVHBjN41mCGfT4MU2fIPz2f1bes5vBebaymlPKcxoT9DcaYe4wxNcaYrcaY0cDR7sH3miFD4Kuv4Ikn4OuvITERnnwS6hxyfbPzGZ3JXJ5J9B+i2TLL3VjtA22sppTyjOOGvTEm5wivveSdco4tIABuvRUKC60Lub/7HZx6Kqw87mVf3xDYIZC4v8eR9m0agR0DWXHRCorGFnGo/JDdpSmlfJxPNkKLiYGPPoIXXoCiIhg2zJrXr3FIs8mOwzuSsTSDmKkxlL9ejivBxfZXt2vLBaVUs/lk2AOIwLhxVthffDFMmQJZWZCXZ3dlnhEQEkC/af1IX5pOaL9Qiq8spmB0AQfLtLGaUqrpfDbs6/XoAW+8AW++ad2amZlpXdCtrra7Ms8IHxpO2ndpDHh0ALs/2012QjZbntmio3ylVJP4fNjXGzPGGuVfcw089JA1tfP113ZX5RkSKPS5vQ8ZyzOISItg9YTVLDtzGQfWHbC7NKWUj3BM2AN07gzPPQeffAKHDsHIkdYF3cpKuyvzjLC4MIZ9PoxBTw+iMrcS11AXmx/bjKnVUb5S6tgcFfb1zj4bVqyASZPgX/+CpCT4+GO7q/IMCRB6TehFZmEmnc/szLrb17H05KXsK3BIf2illFc4MuzBapE8c6Y1lRMWZj2INW4c7LTr1vVp0zy6u9DoUJLeTSL+lXiq11eTm5bLhmkbtLGaUuqIHBv29U4+2bpDZ8oUmDfPegp3/nwbWi7c6/lVGUWE7ld0J7M4k66/7krJvSXkpOWwN1sbqymlfszxYQ9WY7UHHrAaq0VHw69/bTVX27rV7so8o12XdiTMTSDpvSQOVxxm6UlLWXv7Wm2sppT6gV+Efb2UFFiyBB5+2HooKyEBnn/ei6P8adOsBwLEvSpV/fcentKp1+WiLmQVZtHzxp6UPlaKa6iL3V84ZEEApVSLHHWlKjtkZGSYnJyfdWfwitWrYfx4a7GUs86yVsbq18+LBxRp1bmj3Qt3s2r8KqrXVdPzxp4M+NsAgjoddclhpZSPavFKVU43aBAsXGjdrbN4sXXHzsyZUOuQmY/Op1uN1frc0Yets7eSnZDNjvd22F2WUsomfhv2YDVWu+UWq7HaaafB5MnWvflFRV442NSpXtjpsQWGBTLgkQGkLU4jOCqYgosLKLpSG6sp5Y/8Ouzr9e0LH3wAL70Eq1ZZi6Y88ICHG6t5aZ6+MTpmdiQ9J53Y+2Ipf7Oc7Phsts/TxmpK+RMNezcRuOoqKC6GX/4S7r4bMjIgN9fuyjwjoF0AsXfHkpGXQfu49hSPLWbFL1ZQvdkhTYSUUsekYf8T3brBq6/CO+9AebnVSfPOO+GAQ9rQdEjsQNo3aQx4fAAVX1TgSnRR9u8yTJ2O8pVyMg37oxg92pq7v/56eOQRq7HaV1/ZXZVnSKDQZ3IfMldkEpEVwZpb1pB/Rj5Va6rsLk0p5SUa9scQGQnPPAOffQaHD1sXcX/7W9jrkAdU2/dvz7BPhzH42cHsy99HTnIOm/62ibrD2nJBKafRsG+EM8+0Gqvddhv8+9/W+rcffmh3VZ4hIvS8oSdZRVl0Prcz6/+0nryT8ti3XBurKeUkGvaN1KEDPPYYfPstdOwIF15oXdDd4ZBb10N6hZD0dhIJrydQvama3PRcNtyzgbqDOspXygk07JvoxBNh6VK45x547TWr5cJrr9nQWM0LRIRuv+5GVlEW3a7sRsn9VmO1PYv32F2aUqqFNOybISTEamKZm2stfn7FFXDJJbBli92VeUZwVDDxL8Yz9MOh1FbWkndyHmtvW0vtfoc8XqyUH9Kwb4HkZPjuO3j0UWt1rIQEePZZZ4zyAaLOjyKzIJNet/SidIbVWG3XZ7vsLksp1Qwa9i0UFAS3325dwE1JgRtvtBqrrVtnd2WeEdQxiEH/HETKVylIkLD87OWsvGElNRWefLxYKeVtXgt7EekjIl+ISJGIFIrI7711rLYgLg4WLICnn7b65g8dal3QdUpjtciRkWQsy6Dvn/uybc42XAkuyt8p/98GNraDUEodn9daHItIT6CnMWapiEQAucAlxpijthlrzRbH3lRaCjffbPXbycqC2bOtrppOUZlbycobVrJ/2X66/rorA58YSLseIc6Zv1LKh9je4tgYs9UYs9T9fSVQDPT21vHakuhoeO89axnE9eshLc26oHvIIc0mI9IjSHel0296P3b8ZwfZCdkA2lhNqTasVebsRSQWSAWWHOF3E0QkR0RyysvLf/ZeXyUCV15ptVy49FJrliM93ZricYKA4ABiDj3PaYdOZ8SukQBIQIBXV+JSSjWf18NeRMKBN4HJxpifNRowxswyxmQYYzK6du3q7XJaXdeu1gj/3Xdh927rPv0//hGqnNCGZto0MAZTaz149VWHL1kU/hVl3W7UxmpKtTFeDXsRCcYK+rnGmLe8eay27he/sBZJufFG+Pvfrds2v/jC7qo8QwKsNXYzCzLpeFJH1kxcQ/5p+VStcsInmlLO4M27cQSYDRQbYx7z1nF8SadOVm+dBQusn884A266CfY44QHVqVNpH9ue5P8mM/j5wewv2I9rmIuSh0q0sZpSbYA3R/anAFcDZ4hIvvvrAi8ez2eMGgXLl1vTOc8+az2M9d57dlfVQu55ehGh57U9ySzOJOrCKDbctYGlw5dSmV9pb31K+Tlv3o3ztTFGjDHJxpgU95dDekW2XFgY/O1v1hO4J5wAF18Mv/mNtWCKE4T0CCHpzSQS5ydysOwguRm5rJ+yntpqhzx4oJSP0SdobZaVZfXYufdemD8f4uOtC7pOuYux66+6klWURY+re7DpwU3kpuay5xsnzFsp5Vs07NuAdu2sLpp5edaTuGPHWiP90lK7K/OM4BOCGfL8EJL/m0ztgVryRuaxZtIaDu87bHdpSvkNDfs2JDERvvnGarPw+efWXP7TT0OdQ65vnnDOCWQWZNL71t6UPVmGK8nFrk+0sZpSrUHDvo0JDLRWxCoogMxMq+3CGWfAmjV2V+YZQeFBDPzHQFIXpRIQGsDyc5ez8rqV1OzSxmpKeZOGfRvVv7+19u0zz1jTO8nJVivlww6Z+eh0Sicy8jPo+5e+bHtpG9kJ2ZS/6ZCr00q1QRr2bZgIjB9vtVw45xy44w446STrtk0nCAwNpP/0/qTnpBPSK4TCSwspuLSAg9sO2l2aUo6jYe8DeveGd96xlj8sKbF67EydCgcdkokRKRGkZafR/6H+7Hx/J654F1tf2KqN1ZTyIA17HyECl10GxcXWMoj33Wd101y82O7KPCMgKIC+d/Ylc1kmHZI6sOq6VSw/dzkHNh6wuzSlHEHD3sdERcFLL1m98vfuhZNPti7o7t9vd2WeETY4jJQvUxj4z4Hs/W4vriQXpU+UamM1pVrI78J+7ty5xMbGEhAQQGxsLHPnzrW7pGa54AKrsdrNN8OMGdbKWJ9/bndVniEBQu/f9iazIJPIkZGsnbSWvJF57C92yCeaUjbwq7CfO3cuEyZMoKSkBGMMJSUlTJgwwWcDv2NH+Ne/4MsvrbVwzzrLuqBbUWF3ZZ4RGhPK0A+HMuTFIVStrCInJYeSB0uoq3HIgwdKtSKvLUvYHN5eljA2NpaSkpKfvR4TE8PGjRu9dtzWcOCA1XLh0UehWzd46ikYPdruqjzn0PZDrJm0hvLXy+kwrANDnhtCRFqE3WUpZTvblyVsizZt2tSk131J+/bw0EOwZIkV9pdcApdfDtu3212ZZ7Tr3o7E1xJJfDuRmu015Gblsu7P66g9oI3VlGoMvwr7vn37Nul1X1S/9OEDD1i3ayYkWBd029AfcC3S9ZKuZBZl0uPaHmx+eDM5KTlULKqwuyyl2jy/Cvvp06cTFhb2o9fCwsKYPn26TRV5R3AwTJkC+fkweDBccw1ceCE44A8YAII7BzPk2SEkf5qMOWTIPzWf1RNXc7jSIY8XK+UFfhX2Y8eOZdasWcTExCAixMTEMGvWLMaOHWt3aV4RHw+LFsHMmdZF3MRE64KuYxqrnWU1VoueHM2Wp7bgSnSx86OddpelVJvkVxdo/dmGDdYSiJ9+CiNHWitkDRpkd1Wes2fxHlbdsIqqoiq6X92duMfjCI4KtrsspbxOL9CqH+nXD/77X3j+eVixwmqs9vDDDmqsdmInMpZmEHN3DN+/8j3Z8dl8//r32nJBKTcNez8iAtdeazVWu+AC+POfYfhwa27fCQJCAuh3Xz/Sc9MJ6RtC0eVFFI4p5OAWhzQRUqoFNOz9UM+e8NZb1jKIZWVW3/y//hWqq+2uzDPCk8NJW5xG/0f6s+vjXWQnZLN1tjZWU/5Nw96P/epX1ih/7FiYPh1SU+Hbb+2uyjMCggLoe0dfMpZnEJ4Szqrxq1h29jIOrNfGaso/adj7uRNOgBdegI8/hqoqGDECJk2CffvsrswzwgaGkbIghUH/HkRldiWuoS42z9iMqdVRvvIvGvYKgHPPtZZCnDgRnnwSkpLgk0/srsozJEDodVMvMosyiRwVybrb1rH0lKXsL9TGasp/aNirH0REwBNPwFdfQWio9QFw3XWwe7fdlXlGaHQoQ98bSvzceA6sPUBOag4b799I3SGHPHig1DFo2KufGTHCukPnrrusVgsJCdYFXScQEbr/pjtZxVl0/VVXNt6zkdyMXPa69tpdmlJepWGvjig0FB580Oqz06OHdTH30kth2za7K/OMdl3bkfBKAkn/SaJmZw1LT1zKuj+to7ZKG6spZ9KwV8eUmgrZ2Vbwv/++Ncp/4QXnNFbrcnEXsoqy6Dm+J5v/tpmcYTnsXuiQeSulGvBa2IvIcyLyvYgUeOsYqnUEB1tTOvn5Vthfdx2cdx74+BIAPwjqFMTgpwczbMEwjDEsG7WMVTev4vAehzxerBTeHdm/AJznxf37BKcsgwgwZIh18fbJJ6378ZOSrAu6Tmms1nlUZzKXZxJ9ezRbn9lKdmI2Oz/QxmrKGbwW9saYr4Bd3tq/L3DaMogAAQHW7ZkFBf+7J//UU2HlSrsr84zAsEDiHo0j7bs0gjsHs+KiFRSNLeJQ+SG7S1OqRXTO3oumTJlCVVXVj16rqqpiypQpNlXkOTEx8NFHMGeO9RTusGHWvH5Njd2VeUbHrI6k56YTOy2W8jfKcSW42P7qdm25oHyW7WEvIhNEJEdEcsrLy+0ux6OcvAwiWI3VrrkGiovh4outBVOysiAvz+7KPCOgXQCxU2NJX5pOaP9Qiq8spmB0AdWlDmkipPyK7WFvjJlljMkwxmR07drV7nI8yh+WQQTo3h3eeAPefNO6NTMz07qge8AhbWjCk8JJ+zaNAY8NYPdnu3ElutgyawumTkf5ynfYHvZO5i/LINYbM8aa0hk3zlr8PCUFvv7a7qo8QwKFPrf1IXNFJhHpEay+aTXLzlxG1dqq479ZqTbAm7devgJ8BwwWkVIRucFbx2qr/G0ZRIDOnWH2bGtFrEOHrFWxbr0VKivtrswz2g9oz7DPhzHomUFULq0kJzmHzX/Xxmqq7dNlCZXX7Ntn9cn/xz+gTx94+mnr/nynOFh2kNW3rGbnezuJyIxg8HODCU8Kt7ss5Wd0WUJlu/BwmDEDvvkGOnSA88+3pnh2OuTW9ZDeIST9J4mEVxOo3lhNblouG6Zt0MZqqk3SsFded9JJ1h06f/0rzJtnPYU7f74zWi6ICN0u70ZmUSbdLu9Gyb0l5KTlsDdbG6uptkXDXrWKkBC4/37IybGmdH79a6u52tatdlfmGe26tCP+pXiGvj+U2j21LD1pKWtvX0vtfm2sptoGDXvVqoYNg8WL4ZFHrIey4uPhueecMcoHiLowiszCTHrd1IvSx0pxJbvYvUAbqyn7adirVhcUBHfcAcuWWeF/ww1wzjmwYYPdlXlGUMcgBv1rECkLU5AAYdmZy1h14ypqKhzyeLHySRr2yjaDBsEXX8BTT8GSJVZjtZkzodYhMx+Rp0WSsTyDPn/qw9bntuJKdLHj3R12l6X8lIa9slVAANx8MxQWwmmnweTJ1r35RUV2V+YZge0DGfDwANKWpBHcJZiC0QUUXlHIoe+1sZpqXRr2qk3o0wc++ABefhlWr7YWTXngAQc1VsvoSHpOOrH3x7Lj7R1kJ2Szfa42VlOtR8NetRkiMHasNaofMwbuvhsyMiA31+7KPCMgOIDYv8aSkZdB2MAwiq8qZsVFK6jerI3VlPdp2Ks2p1s3eOUV+M9/YMcOq5PmnXc6p7Fah4QOpH6dStyMOCoWVuBKdFH2VJk2VlNepWHvME5aGevii625/BtusG7VTE6GL7+0qZhp0zy6OwkUon8fTWZBJh2Hd2TNb9eQPyqfqjXaWE15h4a9gzhxZazISJg1Cz7/3Fr+8PTT4ZZbYG9rP6B6771e2W37fu1J/iSZwbMHs2/ZPnKSc9j0yCbqDmvLBeVZ2gjNQWJjYykpKfnZ6zExMWx0wOrg+/fDPfdY/XZ69bIaq11wQSsdXMTrT34d3HKQNRPXsOOdHYSnhzNk9hDCh2ljNXVs2gjNDzl9ZawOHeDvf7cWO+/UCS68EK66yprX94pp06yQF7F+rv/ew1M69UJ6hZD4ViIJbyRwcPNBcjNy2XD3BuoO6ihftZyO7B3E6SP7hg4dsta8ffBBa6rniSfgssv+l8se1woj+4Zqdtaw9g9r2f7idsLiwxg8ezCdTurUasdXvkNH9n7In1bGatfOGmDn5kJsLFxxBVxyCZSV2VyYhwRHBRM/J56hHw2ldn8teafksWbyGg7vO2x3acpHadg7iD+ujDV0KHz3HTz6qLU6VkICPPOMFwbhU6d6eIeNE3VeFJkFmfT6bS/KZpaRMzSHXZ/usqUW5dt0Gkc5xtq1cOONsHAhjBplhf6AAXZX5TkViypYNX4VB1YfoMf1PRjw6ACCOwfbXZaymU7jKL8TF2fdovn009b0ztCh8NhjDmqsNjKSjGUZ9L2rL9vmbMOV4KL87XK7y1I+QsNeOUpAAEyYYD2MdeaZcPvtcPLJUFBgd2WeERgaSP8H+5OenU67Hu0oHFNI4WWFHNqujdXUsWnYK0eKjoZ337XaLqxfD2lp1nNRhxySiRFpEaRlp9HvwX7seHcH2fHZbHtxmzZWU0elYa8cS8S6S6e42FoGcdo0SE+H7Gy7K/OMgOAAYu6KISM/g7D4MFaOW8mKC1ZQXaKN1dTPadgrx+vSBebOhffeg927rQXQ//hHqHJIG5oOQzqQuiiVuCfiqFhUgSvJRdk/tbGa+jENe+U3LrrImsu/8UbrSdyhQ62VspxAAoToW92N1U7uyJpb15B/Wj5VqxzyiaZaTMNe+ZVOneDf/7ZCPiAAzjgDbroJ9uyxuzLPaB/bnuSPkxnywhD2F+7HNcxFyUMl1NVoywV/p2Gv/NLpp1sLnt9xBzz7rPUw1nvv2V2VZ4gIPcb1ILMoky6/6MKGuzawdPhSKvMq7S5N2UjDXvmtsDCrT/6SJRAVZfXP/81voNwht66H9Agh8Y1EEt9M5OCWg+Rm5rJ+ynpqqx3y4IFqEg175fcyMiAnB+67D+bPh/h4mDevVfueeVXXMV3JKs6ixzU92PTgJnJSctjzjUPmrVSjeTXsReQ8EVklImtF5M/ePJZSLdGunbXmbV6e9STu2LHWSL+01O7KPCO4czBDnhtC8n+TqauuI29kHmsmaWM1f+K1sBeRQOCfwPlAAnCliCR463hKeUJiInzzDTz+OCxYYM3lP/20tUqWE5xwzglkFmTS+3e9KXuyDFeii13/1cZq/sCbI/ssYK0xZr0x5hDwKjDai8dTyiMCA2HyZFixwlrs/Oabrbt21qyxuzLPCAoPYuDMgaR+nUpgWCDLz1tO8bXF1Oyqsbs05UVBXtx3b2Bzg59LgeE/3UhEJgAT3D8eFBGHdDH5mS6At9ZUagscfX5ffkmXQYOce37MoQtzHHx+zv7/5+DGbOTNsG8UY8wsYBaAiOQ0plWnL3LyuYGen6/T8/NdItKovvDenMYpA/o0+Dna/ZpSSqlW5s2wdwEDRaSfiLQDrgDe9eLxlFJKHYXXpnGMMYdF5Fbgv0Ag8JwxpvA4b5vlrXraACefG+j5+To9P9/VqHNrU8sSKqWU8g59glYppfyAhr1SSvmBNhH2Tm6rICLPicj3Tn1+QET6iMgXIlIkIoUi8nu7a/IkEQkVkWwRWeY+v3vtrsnTRCRQRPJE5H27a/E0EdkoIitEJL+xtyj6EhGJFJH5IrJSRIpF5KSjbmv3nL27rcJq4GysB69cwJXGmCJbC/MQETkV2Ae8aIxJsrseTxORnkBPY8xSEYkAcoFLHPS/nwAdjDH7RCQY+Br4vTFmsc2leYyI/AHIADoaYy6yux5PEpGNQIYxxpEPVInIHGCRMeZZ912PYcaYiiNt2xZG9o5uq2CM+QpwbPMRY8xWY8xS9/eVQDHW09OOYCz73D8Gu78cc1eDiEQDFwLP2l2LahoR6QScCswGMMYcOlrQQ9sI+yO1VXBMWPgTEYkFUoElNpfiUe5pjnzge+BTY4yTzm8G8CfAIa3efsYAn4hIrrs1i5P0A8qB593TcM+KSIejbdwWwl45gIiEA28Ck40xe+2ux5OMMbXGmBSsp8CzRMQR03EichHwvTEm1+5avGiEMSYNq/vuRPe0qlMEAWnAU8aYVGA/cNRrnm0h7LWtgo9zz2W/Ccw1xrxldz3e4v4T+QvgPJtL8ZRTgIvd89qvAmeIyMv2luRZxpgy9z+/B97GmjZ2ilKgtMFfmvOxwv+I2kLYa1sFH+a+gDkbKDbGPGZ3PZ4mIl1FJNL9fXusGwlW2lqUhxhj7jLGRBtjYrH+u1tgjLnK5rI8RkQ6uG8awD29cQ7gmLvijDHbgM0iUt/18kzgqDdGtIWul81pq+AzROQV4HSgi4iUAlONMbPtrcqjTgGuBla457UB/mKM+dC+kjyqJzDHfddYAPC6McZxtyg6VHfgbWs8QhAwzxjzsb0ledzvgLnugfJ64LqjbWj7rZdKKaW8ry1M4yillPIyDXullPIDGvZKKeUHNOyVUsoPaNgrpZQf0LBXqhFEZJyIrHF/jbO7HqWaSm+9VOo4ROQEIAerM6TB6uyZbozZbWthSjWBjuyVakBEMkVkubuPfQcRKQQmYjVA2+UO+E9xTssE5Sdsf4JWqbbEGOMSkXeBB4D2wMtADdqZVfk4Hdkr9XP3YfXAyQAesbkWpTxCw16pn4sCwoEIIBTtzKocQC/QKvUT7mmcV7EWh+gJ3IN1Uba+fexSrAu0jl2BTDmPztkr1YCIXAPUGGPmuTtdfgukAPdjteMGuE+DXvkaHdkrpZQf0Dl7pZTyAxr2SinlBzTslVLKD2jYK6WUH9CwV0opP6Bhr5RSfkDDXiml/MD/A5ZejW0+wSYRAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Choose values between 0 and 6\n",
    "x0 = np.arange(0,6)\n",
    "\n",
    "# Plot the two decision boundaries\n",
    "x1 = 3 - x0\n",
    "x1_other = 4 - x0\n",
    "\n",
    "# Plot the decision boundary\n",
    "plt.plot(x0,x1, c=\"b\", label=\"$b$=-3\")\n",
    "plt.plot(x0,x1_other, c=\"m\", label=\"$b$=-4\")\n",
    "plt.axis([0, 6, 0, 6])\n",
    "\n",
    "# Plot the original data\n",
    "plot_data(X,y)\n",
    "# Set the y-axis label\n",
    "plt.ylabel('x1')\n",
    "# Set the x-axis label\n",
    "plt.xlabel('x0')\n",
    "\n",
    "plt.legend(loc=\"upper right\")\n",
    "plt.title(\"Decision Boundary\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see from this plot that `w = np.array([-4,1,1])` is a worse model for our data. Let's see if you can see this from your cost function implementation as well."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cost for b = -3 :  0.3668667864055174\n",
      "Cost for b = -4 :  0.503680863674846\n"
     ]
    }
   ],
   "source": [
    "w_array1 = np.array([1,1]).reshape(-1,1)\n",
    "b_1 = -3\n",
    "w_array2 = np.array([1,1]).reshape(-1,1)\n",
    "b_2 = -4\n",
    "\n",
    "print(\"Cost for b = -3 : \", compute_cost_logistic_loop(X, y, w_array1, b_1))\n",
    "print(\"Cost for b = -4 : \", compute_cost_logistic_loop(X, y, w_array2, b_2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Expected output**\n",
    "\n",
    "Cost for b = -3 :  0.3668667864055174\n",
    "\n",
    "Cost for b = -4 :  0.503680863674846\n",
    "\n",
    "\n",
    "You can see that your cost function behaves as expected and the cost for `w = np.array([-4,1,1])` is indeed higher than the cost for `w = np.array([-3,1,1])`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### (Optional ) Vectorized version\n",
    "\n",
    "You will now implement a vectorized version of the cost function.\n",
    "\n",
    "The vectorized version of the cost function formula is \n",
    "\n",
    "$$ J(\\mathbf{w},b) = \\frac{1}{m} (-\\mathbf{y}^T \\text{log}(\\mathbf{f_{w,b}}) - (1-\\mathbf{y})^T  \\text{log}(1-\\mathbf{f_{w,b}}))$$ \n",
    "\n",
    "where\n",
    "\n",
    "$$ \\mathbf{f_{w,b}} = g(\\mathbf{X} \\mathbf{w})$$\n",
    "\n",
    "As before, $g$ is the sigmoid function.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "**Exercise**\n",
    "\n",
    "You'll complete the vectorized cost function by implementing the following steps - \n",
    "\n",
    "* First, you'll compute the models prediction $f_{\\mathbf{w},b}(x)$ as shown below\n",
    "\n",
    "   ```\n",
    "    f = sigmoid(X @ w) + b  \n",
    "    #f = sigmoid(np.matmul(X,w) + b  # also works\n",
    "    #f = sigmoid(np.dot(X,w) + b     # also works\n",
    "    total_cost = (1/m)*(np.dot(-y, np.log(f)) - np.dot((1-y), np.log(1-f)))\n",
    "   ```\n",
    "  \n",
    "\n",
    "* Then, you'll compute the cost as \n",
    "\n",
    "  ```\n",
    "  total_cost = (1/m)*(np.dot(-y, np.log(f)) - np.dot((1-y), np.log(1-f)))\n",
    "  ```\n",
    "\n",
    "**Debugging Tip:** Vectorizing code can sometimes be tricky. One common strategy for debugging is to print out the sizes of the matrices you are working with using the size function. For example, given a data matrix $\\mathbf{X}$ of size 6 × 3 (6 examples, 3 features) and $\\mathbf{w}$, a vector with dimensions 3x0, you can observe that $\\mathbf{Xw}$ is a valid multiplication operation, while $\\mathbf{wX}$ is not."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def compute_cost_logistic_matrix(X, y, w, b):\n",
    "    \"\"\"\n",
    "    Computes the cost over all examples\n",
    "    Args:\n",
    "      X : (array_like Shape (m,n)) data, m examples by n features\n",
    "      y : (array_like Shape (m,1)) target value \n",
    "      w : (array_like Shape (n,1)) Values of parameters of the model      \n",
    "      b : (array_like Shape (n,1)) Values of bias parameter of the model\n",
    "    Returns:\n",
    "      total_cost: (scalar)         cost \n",
    "    \"\"\"\n",
    "    m = X.shape[0]\n",
    "    \n",
    "    ### START CODE HERE ### \n",
    "    ### BEGIN SOLUTION ###  \n",
    "    f = sigmoid(X @ w + b)\n",
    "    total_cost = (1/m)*(np.dot(-y.T, np.log(f)) - np.dot((1-y).T, np.log(1-f)))\n",
    "    ### END SOLUTION ###  \n",
    "    ### END CODE HERE ### \n",
    "    \n",
    "    return total_cost[0,0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's check if the output of this function is equivalent to the output of your non-vectorized implementation above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cost computed by non-vectorized version: 0.3668667864055174\n",
      "Cost computed by vectorized version: 0.3668667864055174\n"
     ]
    }
   ],
   "source": [
    "print(f\"Cost computed by non-vectorized version: {compute_cost_logistic_loop(X, y, w, b)}\")\n",
    "print(f\"Cost computed by vectorized version: {compute_cost_logistic_matrix(X, y, w, b)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Expected output**\n",
    "\n",
    "Cost computed by non-vectorized version:  0.3668667864055174\n",
    "\n",
    "Cost computed by vectorized version:  0.3668667864055174\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
